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ABSTRACT 



National assessments of student academic achievement 



in various subjects and grades are conducted each year in France* 
However, there is no formal evaluation of student performance in 
various cros s-curr i culum domains, such as study skills, self-image, 
interest in learning, cooperative attitudes, social competence, or 
civic knowledge and values. This paper presents findings of a 
longitudinal study that examined students 1 growth in these areas, as 
well as in the subjects of French and mathematics. A stratified 
sample of 100 schools was drawn in 6 school districts. In each 
school, a random sample of 80 students entering grade 6 completed 2 
cognitive tests and a rotated form of the cross-curriculum 
instruments. Posttests were administered to the 8,000 students 
twice — after years and then after 4 years in secondary school. 
Questionnaires about the school environment and the learning 
processes were completed by students in grades 6, 7, and 9; 30 
teachers; and the school directors. The paper presents findings from 
phase 1 of the study, which examined students' progress from the 
beginning of grade 6 through the end of grade 7. Findings indicate 
that students' academic progress in French and mathematics improved; 
however, most of their cross _ curricular outputs declined. Girls 
showed improvement in French, civics, study skills, cooperation, and 
social skills. Boys' mathematics and self-concept scores increased. 
The following variables correlated highly with student outcomes: 
positive expectations, school climate, opportunity to learn, time 
management, discipline, and clear rules. Achievement gaps were 
reduced in school settings that included students of different 
socioeconomic (SES) and ethnic backgrounds. High-SES communities 
demonstrated the highest levels of academic effectiveness. Four 
tables are included. (LMI) 
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I. GENERAL FRAME 



While national assessments of the students' academic achievement are done each year in 
France for different subjects and different grades, no comparable information is available about 
a number of cross-curriculum domains, such as students' Study skills, Self image, Interest in 
learning, Cooperative aitUudes, Social competence or Civic knowledge and values. 

Both the school authorities and the families seem to consider these goals as very 
important; and indeed, a top-priority is attached to them in many passages describing the general 
curriculum in France. However, there is no formal evaluation of the students' outcomes for 
these aspects. 

This study, which was granted by the Direction de V Evaluation et de la Prospective of 
the French Ministery of Education, is a first attempt in this area in France. It had three aims: 

a. To develop appropriate instruments for large scale surveys on a set of cross- 
curriculum outcomes. 

b. To assess the students' growth in these domains, and also in two cognitive subjects 
(mother language and maths), from the beginning of grade 6 (where french students 
enter the secondary school) to the end of grade 9 (end of junior secondary school). 

■> 

c. To find whether the attended school has an effect on the pattern of students' 
outcomes, and whether the school characteristics which seem to be important 
determinants of academic outputs (school climate, leadership style, time 
management, teacher expectations, etc.) are also effective for cross-curr^ulum 
outputs. 

A stratified sample of 100 schools was drawn in six school districts. Jn each school, a 
random sample of 80 students entering the grade 6 was administered the two cognitive tests and 
a rotated form of the cross-curriculum instruments. The 8000 students were post-tested twice 
(after tw » years and then after 4 years in secondary school). 

Extensile information about the school environment and the learning processes was 
collected in grades 6, 1 and 9 from questionnaires completed by the students, by a sample of 
30 teachers (10 in each grade) and by the school director. 
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Only the data about the first phase of the study (progress from the beginning of grade 6 
to the end of grade 7) have been analysed until now, and will be presented in this article. 



II. THE INSTRUMENTS 



1 . The two cognitive pretests (Mother language and Maths), administered to the students 
in October 1990, were those included in the National Assessment which takes place each 
year in France for ail students entering the secondary school. The two post-tests 
administered in 1992 were adapted from another national study (Evaluation en cycle 
d' observation, DEP 1992). They contain approximative^ one third of anchor items, 
common to both the 1990 and the 1992 tests. The students who repeated grade 6 
received the same post-test as the others. 

2 . The cross-curriculum instruments were included in five different booklets. Each 
student was administered a rotated form containing two of them; the rotation system was 
designed to obtain a minimum of 20-30 students passing each scale in each school, and a 
minimum of 600-700 students passing each pair of scales in the whole population. 

a. Booklets 1 and 2 (How I work and How I do my homework) contain a series 
of self-describing items and scales about the students 1 study skills, developed by 
the research team in Liege (Lafontaine, 1988, Detheux and Blondin, 1989). 

b. Booklet 3 (What I am like) includes a French version of the Harper's (1982) Self 
Perception Profile, adapted by Pierrehumbert (1987), who added to this version 
of the SPP a Locus of Control scale. 



Booklet 4 (Me and the others) contains a French version of three instruments: a 
Cooperative/Individualistic/Competitive attitude scale (Johnson and Norem- 
Hebeisen, 1979) ; the Children's Assertive Behavior scale (Michelson and 
Wood, 1982); a School Motivation subscale (extracted from the Quality of the 
School Life questionary by Williams and Batten, 1981). 

Booklet 5. (Life in Society) contains a number of items on Civic attitudes and 
values. Some of them were developed by the research team; some others were 
adapted from the LEA Civics Study instruments (Torney and al., 1975). 
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The administration plan was accurately devised to protect the confidentiality of the 
students' answers to the cross-curriculum (and student's questionnaire) booklets, in an 
effort to minimize the possible desirability bias. 

From these different instruments, 16 pretest and post-test scores were obtained, to be used 
as criteria in the multivariate analyses (see Table 1). 

A multilevel model was used to estimate the school parameters to be used as 
'effectiveness' and 'equity' criteria at the school level. 



3. Student, Teacher and School questionnaires* 

The research team developed these instruments keeping in mind the following 
requirements: 

- All dimension explored should have some theoretical or empirical ground in the school 
effectiveness literature (Purkey & Smith, 1983; Good & Brophy, 1986; Creemers & 
Scheerens, 1989; Scheerens,1983); 

- All dimensions should be measured with an estimable (and at least reasonable) 
reliability; 

- As often as possible, a common map should be used for the student, teacher and 
school questionnaires: the same dimensions should be measured if possible by similar 
items in the three instruments . 

The questionnaire items were grouped to construct about 120 student, teacher and school 
composite variables, which were used in descri^ live (univariate) analyses and in the 
school reports. 

ror the multivariate analyses, these variables were aggregate'* at the school level, and 
grouped again, to form some 25 composite indexes, which were used as predictors in the 
regressions (see table 2). 

A more detailed description of the instruments and of the methods used to obtain the 
school environment variables may be found in the research report (Grisay, 1993 1 ). 



1 This document (in French) can be obtained by writing to the French authorities : DEP6A, 58 bd du Lycee, 
92100 Van ves, France. 
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III. RESULTS 



1. General trends in the students' evolution. 

The average results (Table 1) point to a sharp contrast between the academic and most of 
the cross-curricular outputs. While the students' two-years progress is significant in 
mother language and mathematics, as documented by the anchor items in the two cognitive 
tests, only a little improvement appears in Civics and no improvement at all ir: Study Skills. 
The students' attitudes towards school seem to deteriorate: a negative evolution is observed for 
the Interest in School, Perseverance and Locus of Control scores; the score on a 
Verbal Agressivity scale becomes significantly higher after two years. 

The study points to some possible explanations of this pattern of results, especially for 
Study Skills and Attitudes. 

a. Most schools do not offer their students a real opportunity to learn or improve study skills. 
Some of them do (for example, by implementing specific courses in study skills, or a 
tutoring system for homework); in those schools the evolution pattern is significantly bettei 
than in others for these criteria. 

b. A clear decrease in Parent's support for homework (-0.375 standard deviation) is observed 
during the two first years in secondary school. A number of families (not only those with 
low levels of education) probably feel too difficult to keep on helping their children when 
the school subjects become 'harder'. 

Therefore, many students are left with no more (and actually, often less) resources to 
cope with the school requirements at the very moment when a fast increase takes place in these 
requirements. The deterioration of all the scores describing the students' attitudes towards 
school {Motivation, Locus of Control, Perseverance) and the increase in Agressivity is 
probably related to the anxiety deriving from this situation, as evidenced by a less negative (or 
even positive) evolution of attitudes in schools where an effort is undertaken to teach study 
skills and where the students perceive their teachers as 'always willing to help' or 'encouraging 
us to do our best'. 

The Civics attitude scale shows that the only domain where the average student has 
clear opinions when beginning grade 6 is rejection of racism (80-90% of positive answers). The 
answers on items about discrimination against, women improve from grade 6 to the end of 

Effective and less effective junior schools in France - Aletta GRISAY - 5 



ERLC 



6 



grade 7, as well as the understanding of fundamentals rights. But no improvement is observed 
in items about justice; and the students' attitude towards fraud becomes more tolerant. 

When entering the secondary school, the students experience an important change in 
their physical and social environment, which seem to be rather distressing : their pretest scores 
are quite low for all the Self Image criteria. However, they are no more the 'youngest ones 1 
two years later: some 4 improvement in social status took place, which may explain a slight rise 
of the post-test scores in School Perceived Competence and Social Image. But there is not yet 
any improvement in Body Image post-test scores. About 50% of the students would like to be 
thinner, to have a different nose or different hair, or different body shape than theirs... 

Controlling for the pretest scores, a more positive pattern of evolution is observed for 
girls than for boys. Girl's two-years evolution is better in French, Civics, Study Skills, 
Cooperation and Social Skills. Boy's scores in. Maths and Self Concept improve more than 
girl's. 

For most cross-curriculum domains, a quite positive profile appears among students 
from ethnic minorities: compared with the evolution of native students with the same SES, the 
growth scores of immigrated students are better, especially in Civics and Interest in School. 

Students from high SES background have better gain scores in both academic subjects, 
in Social skills and in Civics. 



3. Between-school variance. 

One important finding is that, in the French centralised school system, the between- 
school variance is only moderate in mother language and in mathematics (20% of the total 
variance of the post-tests is between-schools variance, and 16 % of it is explained by the 
pretest scores, leaving only 4 % of the total variance as a 'net' effect of the schools). 

The between-schools variance is even less for all the cross-curriculum criteria (it varies 
from 9 % in Civics to 0.5 % in Social Self image; and it is reduced to a mere 2% in Civics 
and a maximum of 3.2% in Study Skills when controlling for the pretest scores). 
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4. Intercorrelations between the school effectiveness coefficients. 

The school effectiveness coefficients are only moderately intercorrelated. Only one 
school can be considered as effective for all dimensions, and only five schools are ineffective 
for all of them. However, a factor analysis shows some consistencies: schools which are 
effective in French are also likely to be effective in Maths. The effectiveness indices in 
Study Skills, Work Organisation, Perseverance, and Interest in School are intercorrelated; the 
same happens for Civics, Cooperation, Social Skills and Locus of Control. Finally, a modest 
correlation (.30) is found between the equity coefficients in Math and French. 



5. School characteristics associated with the 'effectiveness' and 'equity' 
coefficients. 

The four factor scores derived from this factor analysis {Academic Effectiveness; 
Academic Equity; Effectiveness in Social and Civic domains; Effectiveness in Study Skills and 
Attitudes) were used as criteria in a series of regression analyses at the school level. The 
composite schocl variables described in Tab. 2 were used as predictors. Table 4. presents an 
overview of the major results . 

A number of interesting findings emerged from these analyses: 

a. When examining the zero order correlations, one finds a group of school variables whose 
correlations with effectiveness are quite high, and which are some of the well-known 
effectiveness correlates described by prior studies in other countries {positive expectations, 
school climate, opportunity to learn, time management; discipline; clear rules). 

Some exceptions are Leadership Style, Frequent evaluation of students' progress and 
Staff consensus about Objectives: these composite variables do not seem to have a clear 
relationship with effectiveness in the French context. 

b. One noticeable finding is that a number of variables which are associated with academic 
progress have also a significant impact on some attitude scores. In schools with a good 
climate, high teacher expectations and extensive opportunities to learn, the students are 
more positive in their opinions about their teacher's skills; they report lower levels of 
indiscipline and Agressivity, and their Interest in School scores improve significantly. 
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c. Most of these variables are strongly intercorrelated. Their correlation with the school 
average SES is also very high. Iherefore, when entered in the regression analyses, they 
explain little unique, but quite large joint variance. 

This means that the schools in high SES communities offer to their students a more 
favorable (more stimulating, more orderly, more work-oriented) learning environment . 
For these reasons, the academic effectiveness is higher in advantaged schools than in 
disadvantaged ones, many of which suffer from serious discipline problems, from 
difficulties in time management and, often, from the staffs tendency to lower their 
requirements according to the low initial scores of their students. 

d. Despite these handicaps, the disadvantaged schools seem to perform slightly better than 
the advantaged ones in the Social Skills and Study Skills domains, which originates a 
rather inextricable series of suppressive effects in the regression analyses. When trying to 
explore these complex interactions, it seems that the disadvantaged schools 'benefit' 
more than the others from their very heterogeneous social environment (especially, their 
large percentage of immigrant students) and from their efforts to offer remedial resources 
to their students. Both these characteristics have somewhat positive effects on students' 
affective outcomes in low-SES schools, much more than in high-SES ones. 

e. A positive evolution of the Study Skills score is found in schools where most of the 
teachers use structured instruction practices. Structured instruction is also related to a 
positive perception of the teacher's competence by students and to an improvement in 
Locus of Control scores. 

f . Not surprisingly, Cooperation scores are improved in schools where students work in 
groups. 

g. Some 'promising' variables, describing 'good' teaching practices (such as Student- 
oriented Teaching or Use of School Library or Staffs perception of Coherence) appeared 
to have no correlation (cr even, in a few cases, negative correlations) with most of the 
effectiveness coefficients. These variables appear to be quite instable : actually, most of 
the composites derived from the teacher questionnaires suffer from their extremely large 
Hctween-teachers within school variance. In addition, innovative teaching practices seem 
to be related with lack of coherence (the scores on heterogeneity of practices are 
significantly higher in schools where the teachers say that they use student-orienied 
practices). 
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Homogeneous grouping is negatively related to school equity. The differences in cognitive 
achievement are aggravated after two years in schools where the students are assigned to 
different classes according to their initial performances. 

On the contrary, the achievement gaps seem to be somewhat reduced in heterogeneous 
settings (schools whose students come from different socio-economic a-.id ethnic 
backgrounds). 

Parental interventions seem also to be important, especially for Study Skills and Interest in 
School. When considering the group of students which a) were 'at risk when entering 
grade 6, and b) had nevertheless a positive evolution, one finds that most of them 
benefitted from a continuing support on the part of their families (parent's positive 
expectations, parent's help with homework). 

A common profile is frequently found in the group of schools where the performances are 
particularly poor in all domains. In those schools, all the indicators point to a severe 
disruption of the social rules and of the 'moral contract' between the students and the 
school staff. The students, the teachers and the school director are unanimous to report a 
poor Climate, an extremely poor Discipline and an ineffective Time management. The 
teachers' low Expectations, low Requirements, low Opportunity to Learn scores, and the 
lack of Remedial Resources reveal the resignated attitude of the staff. In these schools, the 
average students'opinions about the teachers are among the most negative in the whole 
sample of schools. 
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Table 1. Cognition and socio-affective evolution 
during the first two years of secondary school 



N : 7400 students (for cognitive scores). 
2800 students (for the other scores). 



Domains 


Scores 


Nb 
items 


Alpha reliability 
1990 1992 


Evolution 
1990-1992 1 


Academic achievement 


Anchor test (French) 
Anchor test (Maths) 


30 
18 


.82 
.79 


.86 
.83 


+ 0.727 
+ 0.482 


Parents'help 


Parents' help with homework 


6 


.74 


.78 


- 0.374 


Study skills and 
attitudes 


Time organisation 
Study Skills 
Perseverance 
Interest in School 
Locus of Control 


16 
18 
12 
14 
9 


.75 
.81 
.67 
.78 
.66 


.82 
.83 
.69 
.84 
.63 


-0.067 
-0.011 

- 0.263 

- 0.479 
-0.197 


Self image 


Academic Competence 
Social Competence 
Body Image 
Sports Skills 


11 

5 
8 
4 


.78 
.67 
.77 
.68 


.84 
.75 
.83 
.72 


+ 0.160 
+ 0.231 
- 0.064 
+ 0.183 


Social Attitudes 


Cooperation 
Competition 

Assertivene^s of Social 

Skills 
Verbal Agressivity 


14 

6 
14 

8 


.80 
.72 
.72 

.57 


.85 
.78 
.76 

.61 


+ 0.150 
-0.129 
-0.184 

+ 0.193 


Ci ic Attitudes 


Civics 


24 


.69 


.72 


+ 0.176 



BEST COPY AVAILABLE 



ERIC 



1 The standardized evolution index is the difference b|t$en the pretest avec the post-test scores, divided by the 
standard deviation of the pretest score. 



Table 2. Context and process composite variables used 
as correlates of school effectiveness 



Composite variables 



Source 



Alpha 
reliability 



1 . Urbain/rural school 

2 . Priority area school 

3 School SES index (composite variable, including Average SES 
average number of siblings, percent of students from immigrant 
families) 

4. School and class size 

5. Students' Perception of Parents' support 

6 Positive Expectations (Composite variable, including: Students' 
perception of parents' expectations, Students' own expectations 
Students' perception of pairs' expectations, Teacher perception of 
average students' motivation) 

7 . Average students' age in grades 6 and 7 

8 Material resources (Composite variable, including Teachers' and 
Director's opinions about the quality of premices, school library, and 
other material resources of the school) 

9. Young Staff (Composite variable : Proportion of teachers ageo less 
than 35 years; Proportion of temporary teachers) 

10. Leadership Style 

1 1 Staffs perception of coherence in Objectives and Teaching Practices 
(Composite variable, including Teachers' and Director's perception of 
coherence, integration, participation to decision, etc.) 

1 2. Success/failure expectations 

13. Opportunity to Learn (Percent of math items whose content was 
'taught to us', according to the students' answers) 

14 Students' perception of 'good teaching practices' (Composite var. : 
Structured Instruction, 'Most teachers want me to learn', Teachers are 
fair') 

15 Student-centered teaching (Composite var.: individualized practices; 
group work; Students' participation in class decisions; innovative 
teaching) 



A 
A 

A/S 

A 

S 

S/T 
A 

T/D 

A 
D 

T/D 

T/D 
S 



Sources : 



A 
S 
T 
D 



Administrative files from the French Ministry of Education. 
Student questionnaiie. 
Teacher questionnaire. 
School Director questionnaire. 
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0.85 

0.66 
0.75 

0.84 

0.92 
0.77 

0.77 

0.83 

0.54 
0.8 

0.78 
0.81 



Composite variables 


Source 


Alpha 
reliability 


16. 


Warmth in teacher/studcm relationships (Composite: discussions on 
uie iirsi name udMS, siuuems nguio anu rcapvi'Muiiiuc:*^ 


s 


0.67 


17, 


High/low requirements 


D 


0.61 


18. 


Homogeneous grouping 


D 


0.64 


19. 


Remedial devices (Composite Study skills courses; students 1 
tutoring; help in school for homework) 


D 


0.3 


20. 


Use of school library 


s 


0.5 


21. 


Clarity of school's rules 


D 




22. 


School's climate (Composite : students' Well Being scale; perception 
of students' climate (by teachers); perception of teachers' climate (by 
Directors); teachers' and Directors' quality of professional life scale) 


S/T/D 


0.72 


23. 


Time to learn and Discipline (Composite: students' perception of time 
on task; rate of absenteeism; students', teachers' and Driectors 
perception of Discipline) 


S/T/D: 


0.87 


24. 


Heterogeneity of the population (SES, ethnicity, age, ...) 


s 


0.77 


25. 


Heterogeneity of the teaching practices (as perceived by the students) 


s 


0.65 


26. 


Heterogeneity of the daily environment (as perceived by the students) 


s 


0.47 



Sources : A : Administrative files from the French Ministry of Education. 

S : Student questionnaire. 

T : Teacher questionnaire. 

D : School Director questionnaire. 
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Table 3. Between-schools variance of the cognitive and socio-affective scores 







Between-School 


Residual Between- 


Domains 


Scores 


variance 


School variance 






(as a percentage of the 


(after controlling for the 






total post-test variance) 


pretest score) 


Academic achievement 


French 


19.9 % 


3.8 % 




Maths 


20.2 % 


*r. 1 /O 


Study skills and attitudes 


Time organisation 




J.Z /o 






o.yj /o 


1 yf 0/ 
1.4 VO 




Pprc^ vpra riff* 


i n ol 

l.U fo 


(\ 1 Of 

U.J 7b 




Interest in School 


1.1 % 


0.8 % 




T <v*ii<: of ("'ontml 


l.U /O 


i .U vo 


Self image 


Academic Competence 


0.9 % 


0.9 % 




Social Competence 


0.5 % 


0.5 % 




Body Image 


2.5 % 


1.7 % 




Sports Skills 


0.5 % 


0.2% 


Social Attitudes 


Cooperation 


2.6% 


1.7 % 




Competition 


1.2 % 


1.2 % 




Asscrtiveness and Social 


4.2% 


1.4% 




Skills 








Verbal Agressivity 


1.8% 


1.5 % 


Civic Attitudes 


Civics 


8.8 % 


2.3 % 
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Table 4. Some school characteristics associated with school 
effectiveness and school equity 



(Standardized regression coefficients) 





Factor 1 


Factor 2 


Factor 3 


Factor 4 


Predictors 


Effectiveness 


Equity 


Effectiveness 


Effectivess 




(French & 


(rrench & 


(Social skills 


(Study skills 




Math) 


Math) 


& Civics) 


& attitudes) 


school intake 










school oho 


1 OA* 

.380* 




Oil 

-2.1 1 




Heterogeneity of social intake 


.358* 


AO A* 

.484* 






Heterogeneity of pretest scores (F & M) 


. 1 1 J 






.308 


Average age of the students 


-.331 


-.356 






Interaction: Age * Social Heterogeneity 




.208 






School Characteristics & 










Processes 










School & Class size 


-.234 


.156 






Material Resources 


.129 


.228* 


-.264* 




Opportunity to Learn (Math) 


.232* 




.215* 




Student-oriented Teaching 


-.169 








Low requirements 


-.121 




.198 




Homogeneous Grouping 


-.225* 


-.437 






Use of School J ibrary 


-.162 


-.747* 


-.28R 


.787* 


Clarity of School Rules 




.188 






Remedial Devices 






.121 


.163* 


Time management and Discipline 






.571* 




Positive Expectations 








.180 


School Climate 








.292* 


Interactions Intake * Processes 










School SES * Parent's Help 


.157* 








School SES * Remedial Devices 


-.185 




-.277 




School SES * Low requirements 


-.2H() 








School SES * School & Class Size 


.121 








Social Heter. * Student-oriented Teach. 




-.121 






Social heter. * Homogeneous Grouping 
Students'age * Homogeneous Grouping 




.418* 








-.262 






Percent Immigrants * Young Staff 








-.089 


Percent Immigrants * School & Class Size 








-.160 


Percent Immigrants * Parents' Help 








.114 


Explained Between-School 


45 % 


34 % 


28 % 


44 % 


Variance 











* : S. at 0.05 
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